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in Chicago Public Schools by Derek Mitchell and Bill Conrad 

Abstract 

The Quality School Portfolio (QSP) Initiative was a partnership among the 
Chicago Public Schools (CPS), the Joyce Foundation, and the National Center for the 
Evaluation of Research, Standards, and Student Testing (CRESST). The project infused 
data-informed, question-driven, and technology-rich processes into school improvement 
planning, teacher action research projects, feeder/receiver school relationships, and 
communication at eight Chicago Public Schools. This paper reports the successes, 
challenges, and recommendations of this initiative in building the culture, and technical 
and organizational capacities of Chicago schools to carry out data-informed 
improvement. 
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Introduction 

Helping schools effectively use data in support of strategic planning has become an 
important mission for many school districts, educational organizations and individual 
educators throughout the nation especially in light of the No Child Left Behind 
legislation. The American Association of School Administrators (AASA) and the 
National School Boards Foundation (NSBF) sponsored a research study conducted by the 
National Center for Research on Evaluation, Standards and Student Testing (CRESST), 
to study how districts use data support tools and data-informed questions to help answer 
key achievement-related questions. (Baker, in press). This study demonstrated that the 
task is extremely challenging and requires the active participation of leadership at all 
levels of the organization. Effectively infusing data informed district improvement into 
school districts also involves building and sustaining both the organizational and 
technical capacities of school districts. 

Research associated Avith district and school action research studies supports the 
underlying assumption that schools can successfully address and inform key district 
questions if they can access and use the right data. The process of accessing and using 
data is a complex, energy intensive and difficult task. Researchers, led by Norm Webb 
from the Wisconsin Center for Educational Research (Mason, 2001), reported on the 
challenges that several Milwaukee public schools encountered in trying to build both 
technical and organizational capacities to use data in support of informing key strategic 
decisions (Mason, 2001). In her research with the Milwaukee public schools. Mason 
identified 6 challenges that school districts face in using data to support school 
improvement. 

1. Cultivating the desire to transform data into knowledge; 

2. Focusing on a process for planned data use; 

3. Making a commitment to acquire data; 

4. Organizing data management; 

5. Developing analytical capacity; and 

6. Strategically applying information and results. 

Despite the difficulties cited in the research, the signing of the No Child Left Behind, 
made use of data a main focus for this country’s schools (U.S. Department of Education, 
2001). And unless districts are prepared to search through file cabinets and what are often 
thousands of CUM cards, database technologies will play an increasingly potent role. 
Technology tools are becoming more commonplace and even necessary for schools and 
teachers who want to use data-informed processes to inform improvement in student 
achievement. (Baker, E. L., & Herman, J. L. in press; Card, S. K., Moran, T., 1983; 
Mitchell, D. 2001). Technology plays a key role in helping school districts and schools 
look at and disaggregate data for determining student academic achievement at various 
levels within schools Technology can support the collection, organization, and 
visualization of frequently collected classroom level assessment data. Research supports 
the effectiveness of technology in collecting student assessment data. Technology can 
support the collection, organization, and visualization of frequently collected classroom 
level assessment data. Research supports the effectiveness of technology in collecting 
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student assessment data. Lee and Eller (2001) reported on the effectiveness of a data 
support tool called the Quality School Portfolio (QSP) in helping schools use a data- 
informed process to achieve their school improvement goals. Of course, implementation 
of this technology must also be routinely reviewed and evaluated to ensure maximum 
usage and benefits (Baker & Herman; In Press). 

This article reports the efforts of 8 Chicago Schools and their feeder/receiver 
schools to use data-informed processes to address key questions related to school 
improvement, teacher action research, feeder/receiver school relationships, and 
communication. The names of the schools have been kept anonymous throughout this 
paper in order to maintain confidentiality. Building school-based technical and 
organizational systems for using data to support key school questions is truly a 
challenging task as the WCER team discovered and as was reported in 
AASA/NSBF/CRESST District Data Use study. This article describes how these 8 
schools and their feeder/receiver schools addressed key questions using data and data 
support tools like QSP. 

The Quality School Portfolio Initiative was a partnership among the Chicago 
Public Schools (CPS), the Joyce Foundation, and the National Center for Research on 
Evaluation, Standards, and Student Testing (CRESST). 8 schools formed the core CPS 
schools involved in the project. The project also worked with many of the feeder/receiver 
schools for these QSP schools. 

Goals of the project included: 

1. Support technology-rich, data-driven, decision-making in the following key areas: 

a) School improvement, 

b) Student academic achievement at the classroom level, 

c) Feeder/receiver school relationships, and 

d) Communication venues within and beyond schools. 

2. Build the organizational and technical capacities of schools to plan, implement, 
and evaluate data-informed school improvement. 

Similar data-informed initiatives were implemented in Los Angeles within the Stuart 
Foundation supported QSP Initiative. The American Association of School 
Administrators (AASA) and the National School Boards Foundation (NSBF) also 
sponsored projects supporting the use of technology-rich and data-informed improvement 
in schools across the nation. This article will discuss and compare themes and evidence 
found in all three projects. Norm Webb and Sarah Mason (1999) worked with schools in 
the Milwaukee school system to foster data-informed improvement. This team also 
sought to build the technical and organizational capacities of schools to use data to 
inform school improvement. They encountered similar successes and hurdles within their 
work. This article describes the successes and challenges of the Quality School Portfolio 
initiative in the Chicago Public School system. 
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Background 

In 1999, the Joyce Foundation awarded a 3-year grant in to the National Center 
for Research on Evaluation, Standards, and Student Testing (CRESST) to carry out the 
Quality School Portfolio Initiative with a small sample of Chicago Public Schools. Dr. 
Derek Mitchell, the director for the project worked with Dr. Gerry Oberman, the Director 
of the Department of Compliance for the Chicago Public School System, to identify 
schools to participate in the project. Schools were selected schools based upon their 
recognized use of data to support school improvement and their technological capacity to 
use QSP. They selected the following four schools as the first participants: School 2, 
School 6, School 8, and School 9. 

The Quality School Portfolio (QSP) Initiative provided schools with the QSP 
data-support tool. This tool allows schools to collect, organize, and visualize ITBS data, 
ISAT data, and locally generated student achievement and demographic data for the 
purpose of school improvement, improvement of teaching and learning at the classroom 
level, improved feeder/receiver school partnerships, and improved communication. 

This project provided intensive staff development support for the schools 
participating in the project. Teams of 3-6 key members from core QSP schools 
participated in eight 3-hour training sessions at the Medill Technology Center. Technical 
facilitators met weekly with the technology support team at each school. The school 
improvement facilitator met bi-weekly with the principal and other school improvement 
team members to plan how to use QSP in the context of school improvement initiatives. 

A teacher action research facilitator implemented data-informed teacher action research 
projects at the schools. 

Online staff development opportunities were provided through the Collaboratory 
Project and E-Project. The QSP Training Initiative project provided support in helping 
schools use third party hardware and software to collect local assessment data. In 
addition to the QSP tool, schools received a variety of additional resources to help them. 
These resources included data parsing systems that allowed schools to prepare data for 
import into QSP as well as Access databases that allowed schools to collect and organize 
student behavior and other data within their schools. The project also planned, 
implemented, and evaluated four dinner meetings with school leadership teams in 
feeder/receiver school clusters to develop a data-informed and technology rich model for 
sharing student demographic and achievement data. 

Additionally, the QSP Initiative in Chicago generated a variety of products that 
demonstrated evidence that schools addressed the project goals. Several samples of 
school based products that aligned with the goals of the project can be found in the 
following table. 

Table 1 

Project Goals and Sample School-based Results 



Project Goal 
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Project Goal 

Support technology-rich, 
data-driven, decision-making 
in the following key areas: 

a) School improvement, 

b) Student academic 
achievement at the classroom 
level, 

c) Feeder/receiver school 
relationships, and 

d) Communication venues 
within and beyond schools. 

Build the organizational and 
technical capacities of schools 
to plan, implement, and 
evaluate data-informed school 
improvement. 



Sample School-based Results 

School 6 2002-03 SIPAAA plan that uses QSP 
reports to support the effectiveness of the Writing 
Intervention Project. 

5-Week QSP Reading achievement reports for 
grades 2-8 that use local reading assessments to 
inform student achievement in reading at School 
7 Elementary 

E-Project QSP Projects for the School 7 

feeder/receiver school cluster 

Power Point presentation made to the LSC at 

School 4 that used QSP reports to document the 

success of a new reading program 

Each school has a technical expert that can create 

school-based QSP projects, reports, and groups. 

E-project Web site schools can use to share QSP 

projects among feeder/receiver clusters. 

Data support team at School 7 can use external 
data collection systems and QSP to produce 5- 
week assessment reports in reading 



Project Activities 

Phase 1 

The four schools that formed the core of the Quality School Portfolio Initiative 
worked with Dr. Mitchell to develop data teams. Data teams consisted of the school 
principal, technologist, and school improvement team members. These teams participated 
in a 3-day summer workshop at the Medill Technology Center in Chicago facilitated 
professional staff from CRESST. These workshops provided data teams with the 
rationale for using data-informed processes within their school improvement initiatives. 
The teams also learned how to use the various functions of QSP to produce reports and 
groups that supported their school improvement work. 

In addition to the summer workshop. Dr. Mitchell visited each of the QSP schools 
on a quarterly basis to help the schools calibrate their use of QSP in the context of their 
school improvement processes. He also worked closely with the technology interns to 
make sure that their work continued to build the technical capacity of school teams to use 
QSP in the context of school improvement. 

The schools received on-site technical support from a team of technology interns 
that were a part of a program called Managing Information System Technologies 
Program (MIST). Dr. Frank Nardine led this program as part of a cooperative effort 
between Governors State University and the University of Chicago. I^ST interns were 
graduate students that had expert technical knowledge. They provided between 10-15 
hours per week of technical support to the QSP Schools. They helped school staff in 
importing student demographic and academic achievement data into QSP. They also 
assisted data teams in learning how to use QSP to produce reports and groups in support 
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of key school improvement questions. MIST technical interns regularly collaborated at 
monthly meetings and also through an interactive web site. 

The interns began their technical support by helping the school technologists use 
parsing programs developed by a senior researcher at UCLA. Student demographic and 
large-scale assessment data (ITBS) could be downloaded from a centralized CPS 
computer called the Si-Mapper system. Data on this system was in 34 lines. Lines of 
data included demographic and Iowa Tests of Basic Skills (ITBS). The interns 
downloaded the lines of data to diskette. They worked with the school technologists to 
use the Excel parsers to organize the lines of data into comma-delimited files that could 
be imported into QSP. Much of the initial work of the interns focused on refining the 
parsers, helping the school teams use the parsers, and also importing student data into 
QSP. 



The interns played an important role in helping schools import Illinois Standards 
Achievement Test (ISAT) data into QSP as well. The National Computing System (NCS) 
Company sent each of the schools a data disk with their school’s ISAT data. The student 
data file existed in a delimited format that was very difficult for most schools to access 
and use. A senior researcher at CRESST developed an Excel parser that organized the 
data into rows and columns with appropriate headers. The MIST interns detected a 
problem with the parser in that it did not accurately represent the writing assessment data. 
One of the MIST interns redesigned the parser so that the narrative, expository, and 
persuasive writing assessment data was appropriately parsed into distinct rows and 
columns. Interns then imported the parsed ISAT data into QSP so that the schools could 
produce ISAT reports and groups within the context of their school improvement 
planning. 

It was not uncommon for the interns to find that schools no longer had their ISAT 
data disks. If the plain label of the disk did not intimidate school staff, trying to open the 
student files with Excel would certainly intimidate all but the most intrepid data savvy 
school staff Many of these disks ended up in the garbage. Our interns contacted NCS to 
obtain replacement disks and were told that it would cost the schools $250 to replace 
each disk. Corporate gouging of local schools is certainly not an uncommon practice 
among companies that provide data to schools. We found that schools participating in the 
AASA/NSBF/CRESST District Data Use Project also experienced difficulty in securing 
large-scale assessment data in a format that could be imported into QSP. 

At the end of phase 1 of the project. Dr. Mitchell developed a Request for 
Proposal for each of the schools. This RFP challenged the QSP Initiative schools to 
develop proposals that used QSP and data-informed processes to achieve specific school 
improvement goals. Each of the schools submitted proposals. Teams of professional staff 
from CRESST reviewed and approved the proposals. Schools received up to $10,000 per 
school to implement their proposals in Phase II of the Project. 

The plan was to ask schools in the Joyce Project to develop proposals for the use 
of data and support tools is in alignment with Going Deep Mode! for data use. The 
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school proposals focus the schools’ use of data and data support tools in the context of 
projects that engage multiple levels of the organization. Table 2 summarizes the school 
proposals. 

A unique component of this plan was the engagement of CRESST researchers in 
the evaluation and critique of the proposals. CRESST researchers developed a 
comprehensive critique of the school proposals that gave the data team the schools 
concrete suggestions for improving their projects. 

Table 2 

Summary of School Data-informed Projects 



QSP School 


Proposal Summary 


School 9 


Recovery/Prevention Program. This program sought funds to 
support an after school program to help tutor freshmen students 
who were at risk of failing the introductory Algebra course. 
Funds were used to pay for a teacher coordinator for the 
program. QSP was used to monitor student grades for freshmen 
students participating in the program. 


School 2 


Principal’s Reading Club. This proposal sought funding for a 
Principal’s Reading Club that promoted reading within the 
school. The principal led an after school program in which 
students who had volunteered for the program would read and 
critically discuss books selected by the principal. QSP was used 
to visualize and compare ITBS reading scores in a pre- and post 
fashion for the Principals Reading club participants and then the 
non-participants. 


School 6 


Data-informed SEPAA. This proposal sought funding to help 
members of the internal review team use QSP in the context of 
helping to answer key school improvement questions in a data- 
informed way. Some of the questions included: 

■ Are all students learning and being served? 

■ How do we as a school know that students are learning? 

■ Where are we as a school now? 

■ Where do we want to go? 

= How will we get there? 


School 8 


Recovery Program. This program sought to use QSP to monitor 
the effectiveness of an Academic Recovery Program in helping 
students who had failed one or more courses at the school to 
improve their academic achievement. QSP was used to visualize 
grade data, ITBS data, and plan test data for students 
participating in the Recovery Program. 
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Phase n Activities 

In addition to the four core schools that participated in Phase I activities, four new 
schools were invited to participate in the second year of the Quality School Portfolio 
Initiative in Chicago. These schools were; School 1, School 4, School 7, and School 3. 
Many of the feeder/receiver schools of the core QSP schools also participated in Phase II 
activities. During Phase II, the original core QSP schools engaged in implementing their 
QSP Projects. The goals for the Phase II component of the project included: 

1. Continue to build the technical and organizational capacities of QSP schools to 
use data informed processes in support of school improvement. 

2. Develop and use a data-informed model to improve feeder/receiver school 
relationships. 

3. Plan, implement, and evaluate data-informed teacher action research projects. 

4. Improve the abilities of schools to use data to enhance communication with their 
communities. 

In order to accomplish Phase II goals, the staff for the project was amplified. A 
project coordinator was hired at % time to help coordinate Phase II of the Project. Two 
former MIST interns provided weekly technical support in helping to build the technical 
capacity of schools to use QSP. A key component of their work was the identification of 
a QSP technical coordinator within each of the QSP schools. A researcher from CRESST, 
worked with School 9 and School 6 schools to help these schools plan and implement 
data-informed teacher-action research projects. Dr. Mitchell continued his role as director 
of the project and visited schools on a quarterly basis. He played a key role in helping to 
facilitate four Feeder/Receiver dinner meetings throughout the year. These meetings 
resulted in a model that CPS schools could use to use QSP projects to share student 
demographic, event and academic achievement data. 

QSP Spring Conference. The activities of Phase II began with a 2-day QSP 
Conference held at the Medill Technology Training Center in Chicago. The conference 
engaged data teams from the QSP schools in a session given by Peter Mich that helped 
participants understand the importance of using schools’ use of data from the national 
perspective. NCREL contributed a presentation on the principles and pitfalls of using 
data in the context of school improvement. 

Participants also attended sessions where they used school-specific scenarios to learn 
how the functions of QSP could be used to help inform school improvement questions. 
For example. School 7 received a scenario that asked them to analyze a data visualization 
of reading subtest performance for their eighth grade students. The scenario challenged 
them to go back to QSP to develop additional data visualizations that would help them 
better understand the reading performance of eighth-grade students. In a subsequent 
session, participants were provided an electronic format to develop their own data- 
informed scenarios and respond to scenario questions. 

In addition to using and developing school-based data-informed scenarios, the QSP 
schools participated in sessions where they learned how to copy QSP tables into Excel for 
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more advanced data visualization and analysis. They learned how to copy QSP reports for 
pasting into SIP AAA documents. They also were introduced to a possible online 
progress report and how to use commercial optical scanning equipment to collect and 
electronically store local assessment data. The participants completed online evaluations 
for all of the sessions and for the conference as a whole. 

Phase II activities provided intensive staff development support for the schools 
participating in the project. Teams of 3-5 school members from the schools participated 
in eight 3-hour training sessions at the Medill Technology Center in Chicago. 

Workshops focused on the mechanics of QSP as well as how QSP could be used within 
the context of school improvement. Technical facilitators met weekly with the technology 
support team at each school. The School Improvement facilitator met bi-weekly with the 
principal and other school improvement team members to plan strategically to integrate 
QSP reports into the school improvement process at each school. A teacher action 
researcher facilitated data-informed teacher action research projects at the schools. 

Online staff development opportunities were provided through the Collaboratory Project 
and E-Project. 

Additionally, the work during Phase II helped schools use third party hardware and 
software to collect local assessment data. Schools used optical reading hardware and 
software from the commercial companies to score and electronically store local student 
assessment data. 

Medill Workshops. Schools sent data teams to attend the eight QSP training sessions 
at the Medill Technology Center in Chicago. During these three-hour workshops, teams 
participated in a variety of hands-on activities where they worked with their own local 
school data and learned how to use the key functions of QSP. These functions included 
working in the database view, creating groups, building reports, and establishing goals. 
The second half of the sessions focused on using QSP in the context of school 
improvement, teacher action research projects, feeder/receiver relationships, and 
communication. School-based scenarios were used to engage teams in the school 
improvement components of the workshops. 

The Medill workshops provided opportunities for school teams to share how they 
used QSP within their schools. School teams often collaborated in sharing the ways that 
they used QSP and other data support tools. For example, the School 1 team adapted the 
School 5 Behavior Access database to build their own system for collecting behavior data 
within their school. Many of the sessions concluded with online evaluations of the 
workshop. Sometimes the sessions ended with narrative evaluations. Results of the 
evaluations were used to enhance subsequent school visits and designing the next 
workshop at Medill. 

The Medill workshops also provided hands-on opportunities to learn how to use the 
key functions of QSP. Part of this hands-on component engaged participants in using 
online tutorials. Data teams used the unique resources of the Collaboratory Web site 
(http://collaboratory.nunet.net). This Web site included a component called the 
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MediaSpace that provided schools with a series of slides and explanations for using key 
functions of QSP. School teams learned how to use this resource at the Workshop and 
could then refer back to the MediaSpace slides when they returned to their school. In 
addition to the Collaboratory Project, school teams used a free Web site called E-Project 
to share documents, messages, tasks, and calendars. School 7 used this site to share a 
Receiver School QSP Project that they sent via E-Project to School 10 and School 1 1, the 
high schools in their area. 

School-based Workshops. The Medill Workshops resulted in additional site-based 
workshops. The CRESST team implemented these workshops at individual schools 
throughout the year. For example, after a Medill QSP Workshop that focused on 
feeder/receiver school relationships. School 10 requested that our team carry out 
workshops for administrators at their school. Our team developed workshops for 
administrators and teachers at School 10 that engaged teams of teachers in the use of QSP 
in the context of teacher action research projects and feeder/receiver school initiatives. 
The workshops engaged teachers and administrators in school-specific scenarios where 
teachers and administrators worked with QSP-generated visualizations of large-scale 
assessment data to begin to investigate feeder/receiver school relationships. 

Data teams at QSP schools also organized a series of mini-feeder school meetings 
among the core QSP schools and their feeder/receiver partners. These meetings sought to 
identify meaningful student demographic, event, and academic data that could be usefully 
shared among the feeder/receiver clusters. These meetings addressed logistical issues 
necessary to share student information among the feeder/receiver clusters. For example, 
the principal at School 7 set up several meetings with administrative staff from School 1 1 
and School 10, their feeder high schools. At these meetings, staff from School 7 shared 
how they used an innovative system for collecting, organizing, and visualizing local 
reading assessment data. The team explained how they used an optical reading system. 
Excel software and QSP to produce 5-week reading assessment reports for use at grade- 
level teacher meetings. The team also made a presentation describing possible reports and 
groups that could be shared among the School 7 feeder/receiver cluster. 

On-site support. The technical support team met weekly with the technology 
coordinators of each of the schools to help build the technical capacity of our schools to 
use QSP in support of the school’s strategic goals. For example, the team worked with 
the reading coordinator at School 7 to help her design and use a system for collecting 
local reading assessment data. They trained her on the use of scanning software to read 
and electronically store individual student reading assessment scores. They also helped 
her develop a system for importing this data into QSP. They worked with her to create a 
variety of reports that she could produce to help inform the teachers and administrative 
staff at School 7 in how successful the teachers in grades 2-8 had been in improving 
student reading comprehension in 5-week intervals. A technology facilitator helped the 
math coordinator at School 1 build similar 5-week assessment systems for the 7“-8th- 
grade math program at School 1 . 
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The project coordinator provided bi-weekly assistance to school improvement teams 
and curriculum teams in helping these teams interpret QSP reports in support of 
improved academic achievement. For example, he recommended that the math team at 
School 9 used the Iowa Algebra Readiness Assessment to pre- and post-test freshmen 
students who had been assigned to the Double Algebra class period at School 9. This 
assessment provided sub-test scores that aligned with key pre-algebra skills. Teachers 
were able to use the results of the assessment to diagnose specific and group needs in 
Algebra. Teachers and administrative staff used the results of the Iowa Algebra 
Readiness Assessment as one measure to gauge the success of the program. 

A researcher from CRESST met quarterly with teacher action research teams from 
School 6 and School 9 to help them design, implement, and evaluate teacher action 
research projects that used QSP. For example, she helped 5* grade teachers at School 6 
develop QSP reports of student ISAT writing results to identify the need to address 
specific areas of weakness in narrative writing. She worked with the team to design best 
practice writing interventions; design and use writing rubrics that aligned with the state of 
Illinois writing standards and performance descriptors. She also helped the team to 
collect, organize, and visualize writing data using QSP. 

QSP Initiative Feeder Task Force Meetings. Derek Mitchell designed four dinner 
meetings throughout the year for data teams from the core QSP schools and their 
feeder/receiver partner schools. The Feeder School Task Force meetings provided 
opportunities for schools and their feeder/receiver partners to meet to improve 
communication about the school improvement process. The meetings used a small group 
discussion process to focus on important questions mutually related to student 
achievement. 

Questions for discussion included the following: 

1. What student information would be most helpful? 

2. How can technology tools like QSP assist us in making sure that we communicate 
student information in a timely and useful marmer? 

3. What structural changes within our organizations need to be made in order to 
facilitate the timely transfer of student information? 

4. What student information from feeder schools is not currently available but would 
be helpful? 

5. What formats and timelines are most helpful in sharing data among feeder 
schools? 

Additionally, each meeting provided an opportunity to highlight the project work of 
one or more of the core QSP Initiative schools. Feedback from each of the meetings was 
summarized and shared with teams by the CRESST researcher. Mini-feeder meetings 
were held at each of the core QSP schools between the dinner meeting dates. The mini- 
feeder meetings provided an opportunity for individual feeder/receiver clusters to focus 
on local questions and issues that related to their cluster. The final QSP Task Force 
meeting in June resulted in a model for sharing student data within feeder/receiver 
clusters. This culminating meeting also provided the opportunity to share highlights of 
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the work of QSP schools throughout the year. The feeder/receiver school model 
developed by the task force can be found in the figure that follows. 



Figure 1 

QSP Feeder/Receiver Task Force Model 



Activities Summary 

The Quality School Portfolio Initiative in Chicago engaged eight Chicago Public 
Schools and their feeder/receiver school partners in a variety of activities to support data- 
informed school improvement. These activities occurred in two phases over a 3-year 
period. During Phase I of the project four core QSP school data teams participated in a 
summer workshop where they learned how to use the data support tool called QSP. 

These schools also received support throughout the year fi'om technology interns and 
quarterly visits fi'om the Project director. School data teams developed Joyce-fimded QSP 
projects where they planned to use data-informed activities to address specific school 
improvement needs within their schools. 

During Phase II of the project, four additional schools participated. Schools 
focused on using QSP and data-informed process to address school improvement, engage 
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in teacher-action research projects, build feeder/receiver school relationships, and 
improve communication. Activities for this phase included eight 3-hour staff 
development meetings at the Medill Technology Center, four feeder/receiver school 
dinners; onsite visits by two technology facilitators, school visits by a teacher action- 
research facilitator; school visits by a school improvement facilitator; and quarterly visits 
by the project director. Data team support also included online support through the 
Collaboratory project. E-project, and Go to My PC remote staff development technology. 

QSP Initiative Evaiuatioint 

At the end of the second year of the study, the director of the project initiated an 
evaluation to study the effectiveness of the project in helping schools within the project 
use technology-rich and data informed processes to improve schools. This section 
describes the evaluation questions, claim statements, evidence collected and conclusions. 

Evaluation Questions 

The evaluation is designed to address two basic questions: 

1 . How can technology-rich, data-driven, decision-making support: 

a) School improvement? 

b) Student academic achievement at the classroom level? 

c) Feeder/receiver school relationships? 

d) Communication venues within schools? 

2. How do schools build the technical and organizational capacities to systematically 
carry out data-informed school improvement from the perspectives of 

a) Staff development? 

b) Technology hardware and software infusion and/or maintenance? 
Evaluation Methodology 

The methodology for the is evaluation uses a qualitative method of identifying 
claim statements that align with the key questions of the evaluation process and then 
using the processes of triangulation and confirming evidence to either support or not 
support the claim statements. The claim statements for this project follow. 



Table 3 

Evaluation Questions and Claim Statements 



Evaluation Question 


Claim Statement 


How can technology-rich, data-driven, 
decision making support: 




School Improvement? 


School improvement plans and activities 
demonstrate evidence of the use of data and 
information to inform key school 
questions. 


Student academic achievement at the 
classroom level? 


Teacher action research projects 
demonstrate the use of data to inform and 
answer questions related to student 
academic achievement. 
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Evaluation Question 


Claim Statement 


Feeder/receiver school relationships? 


School partnerships among feeder and 
receiver schools use data to inform and 
improve student academic achievement. 


Communication within schools? 


School teams use data and information in 
the context of communication with staff 
and community members 


How do sclhools build the technical and 

organizational capacities to 
systematically carry out data-informed 
school improvement from the 
perspectives of: 




Staff development? 


Schools use data and information to 
support staff development initiatives. 


Technology hardware and software 
infusion? 


Schools use technology hardware and 
software to enhance, innovate, or overcome 
misconceptions related to school 
improvement. 



A method of triangulation was used to determine if the evidence supported the 
claim statements. The method of triangulation requires that data from multiple sources 
support the claim statement. (Miles & Huberman, 1984; Denzin & Lincoln, 1994). When 
three distinct sources of evidence supported a claim statement then that claim statement 
was officially supported by the work. The process of using multiple sources of data to 
support a claim statement improves the validity and reliability of interpretations made 
from those claim statements. 

The evaluation also used a confirming evidence approach to determine the extent 
to which the claim statements were supported. Each claim statement was said to have 
been derived or met by confirming evidence, if and when multiple data measures 
supported it. Claim statements that produced conflicting or differing pieces of data 
measures were said to have produced Mixed Evidence, and were deemed inconclusive. 
Disconfirming Evidence was the term used when multiple data measures verified that 
claim statement had not been successfully supported (Rudy, 2001; 1999). The evaluation 
plan integrated triangulation with the confirming evidence approach as the basis of all 
evaluation activities. 

The primary sources of data used to provide the confirming evidence include: 
School QSP Project; School SIP AAA Plans; Power Point Presentations developed and 
presented by school teams; Online evidence; and E-reports prepared by Project 
Coordinator and Technology Facilitators. 

Table 4 describes the participants in the project and the data sources used by these 
participants for the data analysis. 

Table 4 
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Project Partici 


pants and Data Sources 


Participant 


QSP 

Project 


SIPAAA 

Plan 


Power Point 
Presentation 


Online 

Evidence 


E-Reports 


School 

Principal 


X 


X 


X 


X 




School 

Project 

Coordinator 


X 


X 


X 


X 




School 

Technology 

Facilitator 


X 




X 


X 




CRESST 

Project 

Coordinator 






X 




X 


CRESST 

Technology 

Facilitator 










X 



Evidence from the project was organized into an evidence matrix that included the claim 
statements for the project and the specific confirming evidence that either confirmed, 
provided mixed evidence or disconfirmed the claim statement. 

Materials and Procedures 

Sources of evidence used within the triangulation and confirming evidence 
methodology previously described included School QSP Projects; Joyce-funded School 
Projects; School SIP AAA. plans; power point presentations developed and presented by 
school teams; online evidence; and E-reports prepared by the project coordinator and/or 
technology facilitators. 

School QSP Projects. Each of the schools participating in the project produced QSP 
projects that helped inform key school improvement questions. These QSP projects 
contained a variety of reports, groups, and data base components that provided evidence. 

Joyce-funded School Projects. Four of the schools. School 2, School 6, School 8, and 
School 9 participated in applying for and implementing school-based projects where 
schools used Joyce funds to implement data-informed projects designed to address key 
school improvement questions. These projects resulted in artifacts and final reports that 
represent evidence. 

School SIPAAA Plans. Each of the schools produced school improvement plans called the 
School Improvement Plan for Advancing Academic Achievement report or SIPAAA. 
These plans included embedded QSP reports and references. They also included addenda 
with QSP reports that supported the goals and activities of the plan. 
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Power Point Presentations developed and presented by school teams. Many of the school 
teams produced power point presentations that included reports and groups produced by 
QSP. These presentations were made at staff development meetings, community 
academic achievement report meetings, and at the QSP Training and Task force 
meetings. 

Online evidence. The CRESST team and school teams produced online artifacts and 
products that were found on the Col laboratory project and E-project web sites. 

E-reports. Both the project coordinator and the technology facilitators produced regular 
reports that summarized and interpreted their work in the project. These accounts and 
insights represent evidence. 

A matrix was used to organize the confirming evidence for the claim statements. 
For each claim statement, schools that produced evidence for that claim statement were 
recorded in the matrix. Schools that did not provide evidence for that claim statement 
were not included in the matrix. The matrix shown in Table 5 includes descriptions of the 
evidence as being confirming, mixed, or disconfirming. 
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Discussion 

This discussion interprets and elaborates upon the results aligned to the claim 
' statements. The WCER report, the AASA/NSBF/CRESST District Data Use Project, and 
the Stuart report also interpret the work of schools to used data-informed processes to 
improve. Where appropriate this discussion highlights the work from these projects in the 
context of the claim statements for this project. 

Claim Statement 1: School improvement plans and activities demonstrate evidence 
of the use of data and information to inform key school questions. 

Evidence in support of Claim statement 1 confirms that schools within the project 
used data and information to support key school questions. The AASA/NSBF/CRESST 
District Data Use project found that school districts primarily focus on the use of student 
achievement data in support of district improvement. The districts use student 
achievement data to enhance traditional methods for school improvement, iimovate 
school improvement processes, and overcome misconceptions about factors contributing 
or not contributing to school improvement. A similar pattern was found in this project. 
Schools in the project used data to support traditional school improvement processes. For 
example. School 6 used QSP reports to enhance the way that they visualized student 
academic achievement data through the production of a data-informed school 
improvement addendum. School 7 innovated the way that they used local, timelier data 
in the context of school improvement. School 2 used QSP reports to overcome 
misconceptions about the relationships among event data and student failure rates. 

The District Data Use Project also found that school districts beginning to use 
student achievement data tend to focus initially on working with readily accessible large 
scale assessment data and then using data support tools like QSP to visualize that data 
broadly. The QSP Initiative in Chicago mirrored this finding in that schools initially 
collected readily available Si-Mapper data and then used QSP to create comprehensive 
reports, as in the case of School 7 and School 6, that became addenda to their school 
improvement plans. During phase II of the project, schools used data and data support 
tools in a deeper way to build school-based projects designed to address school 
improvement questions. School 9 used QSP and data informed processes to improve the 
math performance for freshmen students at academic risk of failing the Introductory 
Algebra Class. 

While there is confirming evidence that the schools used QSP and data-informed 
processes to address key school improvement questions, these energy-intensive initiatives 
did not always appear in the SIP AAA plans for the schools. School 7, for example, 
expended a great deal of organizational energy in planning and implementing 5-Week 
Reading assessments to students in grades 2-8. Yet this initiative ^d not appear in its 
2002-2003 SIP AAA. The culture of data-informed school improvement many times was 
different than official school improvement process. Schools like School 7 would 
complete the school improvement plan from a perspective of compliance rather than from 
the perspective of actual school improvement within the school. Schools like School 1 
perceived their official SIP AAA as a showpiece and did not wish to share any possible 
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negative results produced through their comprehensive production of academic 
achievement reports with QSP or their more focused local assessment activities. 

A superintendent in New Hampshire reported the importance of the values of 
openness and transparency to data, a spirit of inquiry, and humility were keys to the 
successful use of data to inform district improvement. The QSP initiative schools 
demonstrated these values as it related to real-world local school improvement, but were 
less likely to do so in public accountability documents such as the SIP AAA. The 
enormous accountability pressures on Chicago Public School principals and school staff 
promoted more of a defensive posture as it related to official school improvement. Thus 
schools were less likely to share ‘dirty laundry” publicly in official documents although 
much was going on within the schools in the use of data-informed school improvement. 

As was reported in other QSP Projects, the technology facilitators played a key 
role in helping the schools effectively use data to support school improvement. The 
facilitators in this project helped to build the technical capacity of the schools to use data 
on their own by creating data download procedures, parsing tools, and just-in-time 
support for the use of QSP in the context of school improvement. During phase I of the 
project, schools became dependent upon the facilitators to carry out the technical aspects 
of the use of QSP. However, during phase II the project director and the technology 
facilitator worked with school teams to identify a school technology facilitator that could 
sustain the use of QSP within the school. Phase II training sessions between the CRESST 
technology facilitator and the school technology facilitator truly enhanced the technical 
capacity of the schools to use QSP. 

Support from the project director and the teacher action research coordinator were 
also instrumental in helping schools use data in support of school improvement. The 
project coordinator worked with school improvement teams to help produce QSP reports 
in response to specific school-based questions. The project coordinator also suggested 
additional assessments and possible interventions to support school improvement. For 
example, the project coordinator suggested the use of the Iowa Algebra Readiness test as 
a pre- / post assessment tools to gauge the success of the Algebra lOX Project. The 
teacher action research coordinator played a similar role in helping to facilitate the school 
based and school improvement aligned teacher action research projects. 

Claim Statement 2: Teacher action research projects demonstrate the use of data to 
inform and answer questions related to student academic achievement. 

Evidence for the effectiveness of teacher action research projects demonstrating 
the use of data to inform questions related to student academic achievement is confirmed. 
Teacher action research teams at School 6 and School 9 successfully plaimed and 
implemented data-informed teacher action research projects. As in the District Data Use 
Project, these teacher action research projects catalyzed a process that allowed schools to 
use data in a deep way to support improvement in student academic achievement. These 
initiatives moved schools beyond the collection of readily available large-scale 
assessment data to the collection, organization, and visualization of local assessment data. 
School 6 staff worked with local narrative writing assessment data to support 
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improvement in student narrative writing. The School 9 team collected and visualized 
Term Algebra grades to support the success of the lOx Algebra project. 

Again the technology facilitators played a key role in helping the schools build 
and maintain the technical capacity to use QSP and third party hardware and software 
like the Scantron system to work with the local assessment data. Initially, they carried 
much of the burden for these tasks but they eventually helped train and support local 
technology facilitators in carrying out these tasks. The CRESST technology facilitator 
effectively trained the school librarian to provide the action research teams with the QSP 
reports and groups necessary to support the Algebra lOX project at School 9. 

The teacher action research facilitator played a key role in helping School 6 and 
School 9 plan and implement the action research projects at their school. The teacher 
action research facilitator planned and facilitated meetings and staff development 
opportunities for administrative and teacher staff. She also assisted in the production and 
interpretation of the QSP reports that were made to help inform the project. The 
facilitation process did encounter difficulties though. As described in the vignettes, 
leadership at School 9 came from the principal that made it difficult to fully engage the 
teachers in the project. At School 6, data visualizations that resulted fi-om the local 
narrative assessments and that continued to show room for improvement were not readily 
accepted by the school administration. 

Claim Statement 3: School partnerships among feeder and receiver schools use data 
to inform and improve student academic achievement. 

Evidence supporting the school partnerships among feeder and receiver schools in 
using data to inform and improve student academic achievement is mixed. The School 
7/School 10/School 1 1 Cluster was the most successful in creating and sharing QSP 
Projects and reports to support student achievement. Again as in the District Data Use 
project, the role of school leadership played a key role. The principal of School 7 set up 
meetings with the receiver high schools to plan how the cluster would use QSP Projects 
to support the seamless transfer of 8* grade students from School 7 to the receiver high 
schools. The expertise and leadership of the school technology facilitator was also 
critical in establishing and maintaining this data informed partnership. 

Other clusters like the School 9 also followed a pattern initiated by School 7 by 
setting up mini-feeder meetings with its feeder schools. However, there are over 80 
feeder schools within the School 9 cluster. Feeder schools invited to the mini-feeder 
meetings at School 9 only sent 8-10 students to School 9. This phenomenon made it 
difficult to establish strong feeder/receiver partnerships at schools like School 9 and 
School 8 that had so many feeder school partners. 

In addition, some of the QSP schools had difficulty in engaging their 
feeder/receiver partners. For example. School 4 could not engage its receiver high 
schools to participate in mini-feeder meetings. Other schools like School 2, were not able 
to find the time to engage their feeder schools in mini-feeder school meetings. 
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Our QSP schools demonstrated that they could use the technical system for 
creating and sharing Feeder/Receiver School QSP Projects. This process involved the 
creation of school-specific projects and reports and then uploading these projects to the 
E-project web site. For example. School 7 uploaded its 8* grade student project to E- 
Project and both School 10 and School 1 1 could then download the projects to their 
computers for incoming freshmen data visualization and analysis. The QSP- Web project 
will make this upload and download process unnecessary as the Feeder/Receiver Clusters 
will have instant online connections to each other’s projects. 

Claim Statement 4: School teams use data and information in the context of 
communication with staff and community members 

Evidence confirms the ability of school teams’ ability to use data and information 
in the context of communication with staff and community members. Both School 6 and 
School 4 communicated student academic achievement using QSP reports embedded in 
school improvement plans. School 6 also used QSP reports in Academic Achievement 
presentations made to the community. Core QSP schools also presented their school- 
based QSP projects at Medill meetings and Task Force meetings. The reading 
coordinator at School 7 provided 5-Week Floating Bar charts and Histogram reports to 
teacher teams on a regular basis. 

Maintaining the values of openness and transparency to data, a spirit of inquiry, 
and humility were evidenced in ways that schools communicated data and information to 
their stakeholders. For example, at school improvement meetings at School 6 school 
teams used longitudinal QSP reports to dispel misconceptions that special education 
students’ performance on ISAT might pull down overall lAT scores for the school. 

Yet communication of data and information was not always open and transparent. 
Again at School 6, there was reluctance on the part of the administration to share data 
that was not positive with the teacher action research team. The administration believed 
that the local narrative assessment data might demoralize teachers that were preparing 
students for the ISAT writing test. A compromise was forged between the Teacher Action 
Research facilitator and the administration to share the data and information in grade 
level meetings with teachers. School 1 was also reluctant to share negative performance 
data within their school improvement plan. Again the accountability pressures of CPS 
may account for the reluctance of schools to share negative performance data within the 
context of public documents. 

Claim Statement 5: Schools use data and information to support staff development 
initiatives. 

Evidence confirms that schools use data and information to support staff 
development. Schools shared progress on data-informed projects at regularly scheduled 
Medill Training sessions. School teams would then share these data-informed projects at 
their schools with staff For example. School 4 made a presentation at a Medill Training 
session that highlighted the success of the Reading Program at School 4. This 
presentation was then used by the School 4 team a meetings with the Local School 
Council (LSC). 
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The CRESST team also helped to facilitate QSP presentations to teachers at 
School 7 and its Receiver High Schools. These presentations created awareness about 
how QSP could be used at the classroom level to support student achievement. These 
presentations did not go beyond creating an awareness though. The QSP-Web project 
will be a much more appropriate tool for teachers to use to help inform student 
achievement at the classroom level. The Stand Alone version of QSP is designed to be 
used as a school improvement tool for school administrators rather than a tool to inform 
teaching and learning at the classroom level. The problem of having to continuously 
update the stand-alone became an impediment for the QSP schools to use the stand-alone 
at the teacher level. 

The teacher action research staff development initiatives used QSP and data- 
informed processes extensively. The Teacher Action Research Coordinator used QSP 
reports and groups as the rationale for carrying out the projects. QSP was used to collect, 
organize, and visualize local assessment and grade data that were collected during the 
implementation of the projects. These reports were shared with teachers as part of the 
staff development for the Teacher Action Research Projects. 

The support of the technology facilitators and the project coordinators were 
instrumental in using data and QSP in the context of staff development. A key element 
of this support could be found in the development and use of school-based scenarios that 
engaged teacher and administrator teams in using question-driven and data-informed 
approaches to interpreting school specific problems. The facilitators used these scenarios 
with teachers and administrators at Medill Training Sessions and at training sessions held 
at the schools. 

Data informed staff development also occurred through the use of the 
MediaSpace on the Collaboratory Project. Teachers and administrators could use this web 
site at the Medill Training sessions and back at the school to learn how to implement 
specific QSP functions. Teams did experience some difficulty in using this site at their 
schools sometimes because of the multiple steps required to access the online staff 
development support. 

Claim Statement 6: Schools use technology hardware and software to enhance, 
innovate, or overcome misconceptions related to school improvement. 

Evidence confirms the ability of schools to use technology hardware and software 
to enhance, innovate, or overcome misconceptions related to school improvement. As 
schools evolved from the use of readily available large-scale assessment data on the Si- 
Mapper system and the NCS ISAT data to enhance traditional data analysis activities, 
they began to innovate in ways to collect, organize and visualize local assessment data. 
This process entailed the use of third party technology hardware and software. School 7 
for example, used Scantron hardware and software to collect and organize their 5-Week 
reading assessment data. School 1 used Chatsworth hardware and software to collect and 
organize 5-Week Mathematics Assessment data. 
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The use of third party hardware and software was not without its difficulties 
though. Schools experienced problems in using technology-facilitator parsers to organize 
text data derived from the Si-Mapper system. The Excel parsers prepared the data for 
import into QSP. However, schools learned that they needed to do additional data 
scrubbing by removing commas and cells with no entries. These modifications were 
enough to inhibit the ability of school technology facilitators in some cases to be able to 
use QSP. 

The parsing system used to prepare the NCS derived ISAT data was problematic 
for schools. The Universal Parser developed by CRESST was inadequate in its ability to 
parse the ISAT data. This parser did not adequately separate the Writing assessment data 
into the 18 separate variables for this assessment. The CRESST technology facilitators 
had to assist the school technology facilitators to hand parse the data into the appropriate 
rows and columns for import into QSP. The difficulties of data organization by third 
party companies were also experienced within the Stuart Project and the District Data 
Use Project. Data companies like NCS and Scantron make it extremely difficult to export 
and parse their data in ways that support data use within QSP. To the credit though of 
our school teams, they were able to develop systems to use 3"^^ party hardware and 
software in ways that supported their school improvement initiatives. 

In addition to the third party hardware and software, the technology facilitators 
created data collection systems using the Access database program. These proprietary 
programs supported specific data collection needs that could not be met by QSP alone. 
For example, a technology facilitator created an Access database that allowed teachers to 
record student behavior data in alignment with the CPS Disciplinary Code. Teachers and 
counselors could use the database to create disciplinary reports. The data could also be 
exported and then imported into QSP in order to analyze behavior patterns within the 
school. Both School 5 School and School 1 used the Behavior Access Data Base. The 
technology facilitators also helped schools develop Excel templates to collect local 
assessment data like grades. 

Discussion Summary 

This discussion of the claim statements demonstrates that schools in the QSP 
Training Initiative use QSP and data-informed processes to help inform school 
improvement initiatives, teacher action research projects, feeder/receiver school 
relationships and communication with staff and community members. Working closely 
with CRESST staff, school teams overcome obstacles such as working with convoluted 
third party data sets and sharing projects using online sources such as the Collaboratory 
Project and E-Project. CRESST directors and facilitators play a key role in helping 
schools build the organizational and technical capacities to use QSP and other data 
support tools independently. Schools also demonstrate evidence that they use data and 
data-informed processes in staff development initiatives within their schools. The QSP 
schools are now poised to use the new QSP-Web product to advance data-informed 
school improvement. 
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Barriers to QSP Implementation 

Barriers to the implementation of QSP manifested themselves in the values and 
cultures of the school organization, the technical capacity of the school, and the 
organizational capacity of the school. This section discusses barriers to the 
implementation of QSP from these three perspectives. 

Values and Culture. Recognition of the values of openness and transparency to 
data, a spirit of inquiry, and humility was an important in the District Data Use Project. 
These values also manifested themselves in the QSP Training Initiative. However, when 
these values were not evident, difficulties arose in the implementation of QSP. 

Resistance by the administration of School 6 to share local narrative assessment data with 
teachers made it difficult to move forward in improving narrative writing at School 6. An 
unwillingness of teachers at School 8 to collect, visualize, and interpret student grade 
data made inquiry into the reasons for student failure rates at School 8 difficult. The 
principal at School 9 did not demonstrate humility in the use of data when he collected 
and used data student grade data in a punitive way with his teachers. Finally, the 
accountability culture at the schools mitigated the open and transparent use of data. For 
example, the school improvement team at School 1 was more concerned about finding 
very limited evidence of success in student achievement data rather than recognizing the 
big picture of the data in pointing to the great need to improve student achievement. 

Technical capacity. The Stuart project found that some of the participants in their 
project lacked even basic technical skills. This barrier was also found in this project. Our 
technology facilitators spent a great deal of time training staff on basic keystroke 
maneuvers or in organizing Excel spreadsheets for import into QSP. Some of the school 
QSP team members were Macintosh-oriented and needed training in IBM platform skills. 
Some of the School QSP team members did not have e-mail addresses, which made it 
difficult to engage in communication. 

The stand-alone nature of the QSP data support tool made it difficult for this tool 
to be used in a ubiquitous manner across the school. Some attempts were made in the 
project to provide teachers with the capability of using the tool. However, the need to 
individually update teacher machines with new QSP data became a barrier to the 
successful implementation of QSP at the classroom level. Even keeping administrative 
staff up to date on the latest QSP projects was difficult. This was especially true at 
School 7 and their QSP intervention project. As the intervention status of students 
changed, it became necessary to update 3-5 computers that contained the student 
intervention data. The stand-alone nature of QSP also made it difficult to sustain the 
feeder/receiver school relationships. While some schools demonstrated that they could 
use E-project to share QSP projects, the process required a great deal of effort on the part 
of school staff 

Schools also experienced significant barriers with the use of third party hardware 
and software. Companies like Scantron oversold schools on more hardware then they 
needed to perform the basic functions that they required. These companies also forced 
schools to give up on traditional functionality to migrate to an electronic system. For 




Page 31 ^2 



Using Technology-rich Data Support Tools to Enhance School Improvement Initiatives 
in Chicago Public Schools (Draft) by Derek Mitchell and Bill Conrad 

example, Scantron machines not longer adequately marked student answer sheets with 
the incorrect responses. Steps required to carry out the collection and organization of 
local assessment data were cumbersome. Finally, companies like National Computing 
Systems provided data files that required significant parsing to bring them into a format 
that could be imported into QSP. Companies designed their systems to create 
dependencies on their proprietary hardware and software instead of independence. 

Organizational Capacity. As reported in the District Data Use Project, schools 
faced barriers in building the organizational capacity to use data support tools like QSP. 
Schools are very busy places. Allocating the human resource required to collect, 
organize, and visualize student data was a real challenge for schools. During phase I of 
the project, school staff depended too heavily on our technology facilitators to import 
student data and create reports. During phase II of the project, school staff did assign 
specific school staff to act as the school-based technology facilitators - usually the 
librarian of the school. This was a definite improvement in building the organizational 
capacities of the school. However, the attrition rate of these staff was also very high. For 
example, the technology facilitator spent a great deal of time training the librarian at 
School 8 to use QSP only to have this individual resign from the school. 

Summary. Efforts to support schools in using data support tools like QSP and data 
informed processes were not without difficulty. Team members experienced barriers in 
school culture as evidenced by the tremendous need of schools to use data to demonstrate 
improvement. Barriers were also experienced in building technical capacity. Some 
school staff did not have even basic technical skills. Data companies provided hardware 
and software that was difficult to use. Schools experienced organizational barriers in 
terms of allocating staff to work with school data. 

The Impact of QSP Use and Decision-Making in Schools 

The use of QSP made an impact on decision-making within schools from the 
perspectives of school improvement, teacher action research, feeder/receiver school 
relationships, and communication. Schools in the project used data and information to 
focus their decision-making efforts on issues that could be measured. Measured failure 
rates in freshmen mathematics drove a decision-making process that resulted in the 
Algebra lOX project at School 9. Measured poor performance in reading at School 2 
drove the Principal’s Reading Club Project. 

QSP also helped to engage schools in carrying out teacher action research 
projects. The school improvement team used ISAT data to demonstrate a drop in 
narrative wnting performance at School 6. These QSP reports catalyzed the planning and 
implementation of teacher action research projects to improve student narrative writing 
performance at School 6. 5-Week Reading assessment QSP reports at School 7 
continuously focused teacher attention on improving student reading performance at 
School 7. 

The availability of 8* grade student achievement data by receiver high schools 
made it possible for the high school to make data-informed decisions concerning student 
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placement and seamless academic support. The 8 grade QSP project developed by the 
team at School 7 provided data and information that School 10 and School 1 1 could use 
to support the placement of incoming freshmen. 

The QSP schools were also able to improve the way that they shared the decision- 
making processes at their schools with school staff and the community. School 9 
communicated the success of their decision to implement the Algebra lOx project with 
their Local School Council through the production of QSP-report rich power point 
presentations. School 6 kept its community informed of student academic progress 
through the planning and implementation of data-informed Academic Achievement 
Reports. Schools also communicated their data-informed decision-making processes 
though the integration of QSP reports into their school improvement plans. 

Recommendations 

Based upon the findings of the evaluation study and the challenges identified, this 
section identifies key recommendations aligned to the goals of the project. 

Goal 1: Support technology-rich, data-driven, decision-making in the following key 
areas: 

School improvement 

Recommendation 1 : Promote a school climate and culture that fosters as spirit of inquiry, 
openness and transparency to data, and humility. 

Recommendation 2: Insure strong data-informed and question driven elements into the 
district/school improvement process. 

Recommendation 3: Insine that at least one staff member can effectively use data support 
tools like QSP in support of school improvement. 

Student academic achievement at the classroom level 

Recommendation 1 : Build and maintain the technical capacity to collect, organize and 
visualize large-scale and local assessment data in ways that support student achievement 
at all levels within the school. 

Recommendation 2; Provide ubiquitous and easy access to student achievement, 
demographic, and event data for school administrators, teachers, and parents. 

Feeder/receiver school relationships 

Recommendation 1 : Establish data-informed partnerships with feeder/receiver schools. 
Recommendation 2: Use the model developed by the QSP Task force to share student 
data among the feeder/receiver schools. 

Communication venues within and beyond schools 

Recommendation 1 : Plan and implement regular communication opportunities with key 
stakeholders to share products and results of data informed school initiatives. 
Recommendation 2: Plan and implement both technology-rich and traditional 
opportunities to share data informed school improvement results. 

Recommendation 3: Plan and implement a process for celebrating and rewarding the 
successful use of data to inform school improvement. 
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Goal 2: Build the organizational and technical capacities of schools to plan, 
implement, and evaluate data-informed school improvement. 

Recommendation 1 : Provide ongoing school-level staff development support for using 
data support tools, statistical tools, and teaching and learning opportunities to achieve 
specific school strategic goals. 

Recommendation 2: Provide just-in-time staff development support for school-level 
teams engaged in school-level improvement projects. 

Recommendation 3: Develop online opportunities to staff develop school staff in the use 
of data-informed processes for school improvement. 

Next Steps 

In 2001, CRESST received a $2 Million grant from the Department of Education 
to develop an online data support tool that provided the unique features of the QSP stand- 
alone product and also provided expanded capabilities for use of the tool at multiple 
levels within school districts. QSP Web 4.0 collects, organizes, and visualizes student 
academic, demographic and event data at the classroom, school, and district levels. New 
features of QSP Web include an online grade book for teachers, electronic portfolio 
capability, extensive communication capabilities, and a parent reporting function. 

The Chicago Public School System is uniquely positioned as a pioneering partner 
with CRESST and the Joyce Foundation to pilot QSP Web in Area 20. The QSP Web 
Project in Chicago will provide a great opportunity to build upon the successes of the 
Joyce-funded QSP Initiative. The Area Instructional Officer for Area 20, played a key 
role in effectively using the QSP stand-alone version in key school improvement and 
teacher action research projects when he was the principal of School 9. He was a 
successful leader in the Joyce QSP Training Initiative and he will be an excellent 
instructional leader for the proposed QSP Web Pilot in Area 20. 

Conclusion 

The vision of the Quality School Portfolio Training Initiative in Chicago was to 
provide local schools with the data support tools and processes to use data and 
information in support of student academic achievement. By fostering data informed 
school improvement, teacher action research projects, feeder/receiver school 
relationships, and data-rich communication, the project has helped eight schools and their 
feeder/feeder partners achieve the vision. 

The QSP-Web Pilot in Chicago represents a natural extension of the QSP 
Training Initiative. It will bring the power of data-informed decision-making to a much 
wider audience including administrators, teachers, and parents. It will combine the power 
of quantitative data analysis with the qualitative collection and sharing of student work. 
QSP-Web is a tool that will truly support the improvement of student achievement in 
Chicago Public Schools. 

The success of the project can be summarized in the words of Victoria Bernhardt 

(1998), 
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“Data provide the power to make good decisions, work intelligently, work effectively and 
efficiently, change things in better ways, know the impact of our hard work, help us 
prepare for the future, and know how to make our work benefit all children.” 
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